This is an R Markdown Notebook
Deep Learning… This lecture is dedicated to the implementation of Deep Learning models into for our Data
The idea will be to use one particular feature e.g. Tubing Process, Impedance to play with. We will not yet implement this in Shiny but rather focus to prepare our data, fit and test the model
Let’s get our time - series data as a dataframe first…
library(tidyverse)
library(plotly)
# ============= READ DATA =================
# Read our small data ...
DF_Data_Recent <- readRDS("DF_Data_Process_Recent.data")
DF_Equipm <- read_csv("DF_EquipmData.csv")
# data frame containing Event Names
DF_EvCode <- read_csv("DF_EvCodeDataProject.csv")
# Data manipulation and saving to the DF_TEMP
DF_TEMP <- DF_Data_Recent %>%
# join to decode equipment serial number
inner_join(DF_Equipm, by = "IDEquipment") %>%
# join to decode Event Code meaning
inner_join(DF_EvCode, by = "EventCode") %>%
# select only column needed
select(StartDate, Name, AnalogVal, EventText)
Then I will plot my data for all 4 machines as just to remember how it looks like…
# creating human readable data and visualize them
DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
ggplot(aes(x = StartDate, y = AnalogVal, col = Name)) + geom_point()+facet_grid(~Name)
Looking on the chart above I can see that machine #1 seems to be the best. Let’s assume that I take this for granted using my ‘domain’ knowledge. In case you just a consultant or just a Data Scientist, you may want to find and consult this ‘person’. For this reason, I will take that data from Machine #1 as a reference to build my Deep Learning Model.
Following chunk of code will extract this dataset with a filter() function.
# extracting only one machine
DF_M1 <- DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
filter(Name == "Machine #1") %>%
select(StartDate, AnalogVal) %>%
arrange(StartDate)
head(DF_M1)
Now we need to transpose our data from long to wide structure and to fit it to matrix! For the moment, we will ‘forget’ the StartDate values and simply parse the values into matrix of dimension say 150 columns and 20 rows… Of course one need to recover some basic R skills for that :) if not use stackoverflow… (How to turn a vector into a matrix in R?)[https://stackoverflow.com/questions/14614946/how-to-turn-a-vector-into-a-matrix-in-r]
But in order to make things more clear for you (and for me) I will start simple. I will limit the output to 50 pieces of values to have more clear understanding on the process we do.
NOTE: Play with the code yourself to better understand every step!
DF_M1 <- DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
filter(Name == "Machine #1") %>%
arrange(StartDate) %>%
select(AnalogVal) %>%
head(50) %>%
t() %>% # this brings us a matrix
matrix(nrow = 5, ncol = 10, byrow = TRUE) # transforming that into matrix size 5rows and 10columns
DF_M1
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] 50 44 43 49 43 50 44 50 44 50
[2,] 44 51 45 51 43 50 44 50 44 50
[3,] 43 49 43 49 45 51 44 51 45 51
[4,] 44 43 49 43 49 43 49 43 52 45
[5,] 51 44 50 44 50 44 45 51 44 50
Notice to set byrow argument to TRUE. This way we will keep track on how our data are populated. Every row will contain about 150 observations of the Time-Series data. This will be equivalent of known time period.
Coming back to our data. Let’s complete this procedure for entire dataset. Let’s use the recipe to do a final transformation:
DF_M1 <- DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
filter(Name == "Machine #1") %>%
arrange(StartDate) %>%
select(AnalogVal) %>%
head(3000) %>%
t() %>% # this brings us a matrix
matrix(nrow = 20, ncol = 150, byrow = TRUE) # transforming that into matrix size 20x150
Wonderful, let’s try to see! our new object as an image!!!
Let’s use plotly 3D graph to explore what we have got!
plot_ly(z = DF_M1, type = "surface")
Values from Machine 1 used for Training
This should be good enough to fit our deep learning model.
As mr Bill Gates once said: “I would rather hire a lazy person as he/she will find the easiest way to solve a problem…” we must be lazy! For that I will already now think about writing a function that will ‘prepare’ my test datasets. Take a moment to study this:
# build function converting time series data to matrix
to_matrix <- function(x, filter_Event, filter_Machine, n_cols) {
### PURPOSE: Transform Time Series Column of the dataframe to the matrix
# with specified number of columns. Number of rows will be automatically
# found and remaining data points discarded
# # Uncomment variable to debug function
# filter_Event <- "Tubing Process, resistance Ohm"
# filter_Machine <- "Machine #4"
# x <- DF_TEMP
# n_cols <- 150
# get intermediate object and dimension
Step1 <- x %>%
filter(EventText == filter_Event) %>%
filter(Name == filter_Machine) %>%
arrange(StartDate) %>%
select(AnalogVal)
# find number of rows of data frame
nrows <- Step1 %>% nrow()
# find the number of row in a matrix (Whole Rows), the value will have decimals...
WN <- nrows/n_cols
## extract the whole number uncomment for debug/test
# WN <- 19.2
# WN <- 19.8
if((WN - round(WN)) < 0){WN <- round(WN) - 1} else {WN <- round(WN)}
# find number of rows to extract data
n <- n_cols * WN
# extract relevant matrix
Step2 <- Step1 %>%
head(n) %>% #only use whole number to avoid errors
t() %>% # this brings us a matrix
matrix(nrow = WN, ncol = n_cols, byrow = TRUE) # transforming that into matrix size 20x150
# return the result of the function
return(Step2)
}
This function can definitely be easily reused for our ShinyApp. It can be used to generate dataset for any other machine event. Code snippets below will use this function to generate ‘Test’ datasets.
A. For machine 2:
DF_M2 <- to_matrix(DF_TEMP,
filter_Event = "Tubing Process, resistance Ohm",
filter_Machine = "Machine #2",
n_cols = 150)
B. For machine 3:
DF_M3 <- to_matrix(DF_TEMP,
filter_Event = "Tubing Process, resistance Ohm",
filter_Machine = "Machine #3",
n_cols = 150)
C. For machine 4:
DF_M4 <- to_matrix(DF_TEMP,
filter_Event = "Tubing Process, resistance Ohm",
filter_Machine = "Machine #4",
n_cols = 150)
And we can also visualize that to confront:
For Machine 2
plot_ly(z = DF_M2, type = "surface")
Values from Machine 2 used for Test
For Machine 3 - our candidate for anomaly detection!
plot_ly(z = DF_M3, type = "surface")
Values from Machine 3 used for Test
For Machine 4 - Another candidate for anomaly detection!
plot_ly(z = DF_M4, type = "surface")
Values from Machine 4 used for Test
Launch the deep learning machine again…
# to load the library
library(h2o)
# to initialize the 'machine'
localH2O = h2o.init()
H2O is not running yet, starting it now...
Note: In case of errors look at the following log files:
C:\Users\fxtrams\AppData\Local\Temp\RtmpwjnYF6/h2o_fxtrams_started_from_r.out
C:\Users\fxtrams\AppData\Local\Temp\RtmpwjnYF6/h2o_fxtrams_started_from_r.err
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
Starting H2O JVM and connecting: . Connection successful!
R is connected to the H2O cluster:
H2O cluster uptime: 2 seconds 104 milliseconds
H2O cluster version: 3.14.0.7
H2O cluster version age: 24 days
H2O cluster name: H2O_started_from_R_fxtrams_pfu905
H2O cluster total nodes: 1
H2O cluster total memory: 1.77 GB
H2O cluster total cores: 4
H2O cluster allowed cores: 4
H2O cluster healthy: TRUE
H2O Connection ip: localhost
H2O Connection port: 54321
H2O Connection proxy: NA
H2O Internal Security: FALSE
H2O API Extensions: Algos, AutoML, Core V3, Core V4
R Version: R version 3.2.5 (2016-04-14)
Then we will download the datasets into h2o. Remember H2O is just operated from R but it’s a computer besides!
# Import train data into the H2O cluster
train_M1 <- as.h2o(x = DF_M1, destination_frame = "train_M1")
|
| | 0%
|
|================================================================================================================| 100%
# Also import our test datasets for Machines 2 and 3...
test_M2 <- as.h2o(x = DF_M2, destination_frame = "test_M2")
|
| | 0%
|
|================================================================================================================| 100%
test_M3 <- as.h2o(x = DF_M3, destination_frame = "test_M3")
|
| | 0%
|
|================================================================================================================| 100%
test_M4 <- as.h2o(x = DF_M4, destination_frame = "test_M4")
|
| | 0%
|
|================================================================================================================| 100%
Now, once we know how our data looks like we can start to do our Anomaly Model.
# Train deep autoencoder learning model on "normal"
# training data, y ignored
normality_model <- h2o.deeplearning(
x = names(train_M1),
training_frame = train_M1,
activation = "Tanh",
autoencoder = TRUE,
hidden = c(50,20,50),
sparse = TRUE,
l1 = 1e-4,
epochs = 100)
|
| | 0%
|
|================================================================================================================| 100%
Let’s use this model on our training dataset…
# computer error of the model
h2o.anomaly(normality_model, train_M1) %>% as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")
# visually see it
test_recon_M1 <- h2o.predict(normality_model, train_M1) %>% as.matrix()
|
| | 0%
|
|================================================================================================================| 100%
plot_ly(z = test_recon_M1, type = "surface")
Now we will try to use test datasets from machine 2.
# Compute reconstruction error with the Anomaly
# detection app (MSE between output and input layers)
h2o.anomaly(normality_model, test_M2) %>% as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")
What a mess you would say - all observations are anomalous?? Yes! In fact this is correct. Indeed, sometimes AI lead to such a discoveries… In reality we observe that Machine #2 is completely different from Machine #1…
And for Machine 3
h2o.anomaly(normality_model, test_M3) %>% as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")
Machine 3 has only anomalies at certain periods. There are general good performance while at approximately 10% of time we have a periods of anomalous behaviour…
Machine 4 You may try to practice with Machine 4!
h2o.anomaly(normality_model, test_M4) %>% as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")
Machine 4 shows anomalies only at periods 5 and 8. Then the behavious stabilizes…
To use our model in our ShinyApp we will save it…
if(!file.exists("www/tmp/normality_model.bin")){
h2o.saveModel(normality_model, "www/tmp/normality_model.bin")
h2o.download_pojo(normality_model, "www/tmp", get_jar = TRUE)
}
And let’s not forget to switch off our cluster!
h2o.shutdown(prompt= FALSE)
[1] TRUE
in this lecture we have performed:
our next step will be to repeat the procedure but on our machine data.
example from: (https://dzone.com/articles/anomaly-detection-with-deep-learning-in-r-with-h2o)[https://dzone.com/articles/anomaly-detection-with-deep-learning-in-r-with-h2o]
More reading: (https://dzone.com/articles/the-basics-of-deep-learning-how-to-apply-it-to-pre?fromrel=true)[https://dzone.com/articles/the-basics-of-deep-learning-how-to-apply-it-to-pre?fromrel=true]
And: (https://shiring.github.io/machine_learning/2017/05/01/fraud)[https://shiring.github.io/machine_learning/2017/05/01/fraud]
paper: (https://arxiv.org/abs/1701.01887)[https://arxiv.org/abs/1701.01887] In this lecture we would explore the ‘technology’ on the sample and try to do this in 10 min lecture!